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Abstract 

Detector control systems (DCS) include the read out, control and supervision of hardware devices as well as 
the monitoring of external systems like cooling system and the processing of control data. The implementation of 
such a system in the final experiment has also to provide the communication with the trigger and data acquisition 
system (TDAQ). In addition, conditions data which describe the status of the pixel detector modules and their 
environment must be logged and stored in a common LHC wide database system. 

At the combined test beam all ATLAS subdetectors were operated together for the first time over a longer 
period. To ensure the functionality of the pixel detector a control system was set up. 

We describe the architecture chosen for the pixel detector control system, the interfaces to hardware devices, the 
interfaces to the users and the performance of our system. The embedding of the DCS in the common infrastructure 
of the combined test beam and also its communication with surrounding systems will be discussed in some detail. 
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1. Introduction 

For the first time, segments of all ATLAS subde- 
tectors were integrated and operated together with 
a common Trigger and Data Acquisition (TDAQ) , 
close to final electronics and the Detector Control 
System (DCS) at a CERN test beam. 
During this test and certainly in the future exper- 
iment the overall aim of the DCS was and is to 
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guarantee a reliable physics data taking and a safe 
operation of the detector. This is done by moni- 
toring and controlling the DCS hardware, reacting 
to error conditions, providing several user inter- 
faces and maintaining the communication to com- 
mon infrastructure of the ATLAS experiment like 
TDAQ or database systems. Especially the com- 
munication between TDAQ and DCS is of major 
importance for the operation of the pixel detector 
as tuning of the read out chain requires access to 
both systems in parallel. 



Preprint submitted to Elsevier Seicnce 



2 February 2008 



2. Experimental Setup 



vise ttie system in-house developed software tools 
have been used. 
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Fig. 1. Pixel detector module 

The pixel detector module, shown in figure 1, is 
the smallest unit pixel DCS can act on. It consists 
of a silicon sensor and 16 front end chips as well 
as a Module Controller Chip (MCC) gathering hit 
data and servicing trigger requests. 
Every detector module is connected to an opto- 
board which converts the electrical data signals 
transmitted from the modules to an optical signal 
for transmission to the off-detector electronics via 
optical fibres. In parallel it receives optical signals 
from the off-detector electronics and converts these 
to electrical signals for distribution to the modules. 
The off-detector component Back Of Crate card 
(BOC) which serves as the optical interface be- 
tween the Read Out Driver (ROD) and the opto- 
board [1] is controlled by TDAQ while DCS takes 
care of the on-detector component optoboard. 
To operate six pixel detector modules as a part of 
the whole pixel detector its DCS provided various 
equipment at the combined test beam (shown in 
figure 2). For more details about the design of the 
pixel DCS please refer to [2]. 

General purpose 10 devices for the read out of the 
DCS hardware (ELMB ^ ) developed by ATLAS 
DCS group, a home made supply system of three 
low voltage sources together with a reset signal to 
operate the optoboard (SC-OLink^), a regulator 
system for protecting the FE chips of the detector 
modules, developed by INFN Milano, a high volt- 
age source for the depletion of the sensors and also 
temperature and humidity sensors have come into 
operation. To integrate the hardware and to super- 




Fig. 2. Pixel DCS test beam set up 



3. Detector Control System 

The ATLAS detector is hierarchically orga- 
nized in a tree-like structure into sudetectors, 
sub-systems, etc.. This has to be reflected in the 
design and implementation of the DCS. 
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Fig. 3. Detector control system 

Therefore DCS is organized in three functional 
layers: 

- the global control station which e.g. provides 
tools for the overall operation of ATLAS, 

- the subdetector control station which e.g. pro- 
vides full stand-alone control capability and syn- 
chronises the supervision of all subsystems be- 
low and 

- the local control station which e.g. reads data 
from the DCS hardware. 
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The core of the software is based on the commer- 
cial Supervisory Control And Data Acquisition 
(SCADA) package PVSS ^ . PVSS allows to gather 
information from the DCS hardware and offers 
the implementation of supervisory control func- 
tions such as data processing, alert handling and 
trending. It has a modular architecture based on 
functional units called managers. Applications can 
be distributed over many stations on the network 
which defines a distributed system [3] . 



3.1. Distributed System 
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Fig. 4. Distributed system 

At the combined test beam we embedded three 
PVSS stations as a distributed system based on 
a fidl detector simulation as shown in figure 4. 
This test demonstrated successfully the partition- 
ing over several computers and their interconnec- 
tion in a common environment. 

3.2. Software tools 

The software of the pixel DCS consists of several 
subprojects such as tools for the implementation 
of the DCS hardware in the software environ- 
ment and the configuration in an automated way, 
tools for combining all information concerning one 
detector module in a flexible way (see figure 8, 
last page) and also graphical user interfaces. For 
example figure 8 (last page) shows the System 
Integration Tool (SIT) which follows the detec- 
tor hierarchy and therefore maps the real cabling 
structure into the software. 

^ Prozefi- Visualisierungs und Steuerungs- Softwcire, 
ETM, Austria 



All these software tools were used at the combined 
test beam and the experience now helps to develop 
advanced tools for the experiment. 



4. DAQ-DCS Communication 

TDAQ and DCS are controlled by finite state 
machines which consist of different states and tran- 
sition functions which map a start state to a next 
state. Both systems are independent while TDAQ 
has the master control during detector operation. 
This means that the TDAQ finite state machine 
has to be able to cause transitions in the DCS finite 
state machine. Further more TDAQ applications 
have to transfer selective data to DCS as well as 
DCS must make required data available to TDAQ. 
Nevertheless TDAQ must be informed about state 
conditions. 

To cover all the required transfers, the DAQ-DCS 
Communication (DDC) software [4] has been de- 
veloped by the ATLAS DCS group (see figure 5). 




Fig. 5. Schematic of DDC 

DDC was set up in pixel configuration by the 
authors and was running for four months in the 
combined environment. During this time the pixel 
specific DDC application was tested intensely. 
Concerning the command transfer, wc were able to 
show that the used pixel DCS finite state machine 
reacted in a well defined way on TDAQ transitions. 
Additionally pixel DCS directly computed actions 
via DDC in response to three TDAQ transitions 
at the combined test beam. Further more the pos- 
sibility to set DCS hardware with TDAQ applica- 
tions without changing the TDAQ state was tested 
successfully. 

Regarding the data transfer, DCS visualised 
data like temperatures, depletion and low voltages 
of the detector modules or the states of DCS hard- 
ware for TDAQ while DCS received data from 
TDAQ like the status of TDAQ services or run 
parameters. Especially the run number was used 
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for storing run relevant DCS data. In combina- 
tion with a shown dynamical integration of more 
transfer data this was done very efficiently at the 
combined test beam. 

For the message transfer wc built up a DCS finite 
state machine to monitor the parameters of the 
detector modules and to generate corresponding 
states. Pixel DCS sent messages with severity flags 
which were read by the shifter during data taking. 

Performing timing studies, certain DCS actions 
were connected to TDAQ transitions. For the rea- 
son mentioned above, the interconnection between 
on- and off-dctcctor parts of the optical link is of 
special interest. Thus the setting of the reset signal 
of the SC-OLink which allows a slow and controlled 
recovery of the delay control circuit was linked to 
the transition 'LOAD' (see figure 6). 




Transition 



Fig. 6. Required time for various transitions 

The total time for a transition is composed of the 
time of the DDC process and the time of the DCS 
process. For the above example we measured 50 ms 
for the DDC and around 5 s for the DCS process. 
Due to these measurements we were able to opti- 
mise the control code during the test beam period 
together with changes in the hardware properties. 

To verify the full functionality of DDC by the 
shifter during the experiment, additional tools for 
data analysing (see figure 9, last page) are inserted 
in the structure of the pixel detector control system 
which did not effect the normal operation. Check- 
ing the command transfer is done by switching and 
setting any number of virtual power supplies while 
checking the message transfer is done by simulat- 
ing differently weighted temperatures of a detec- 



tor module and sending corresponding messages 
with severity flags. Reviewing the data transfer, 
one could observe from the TDAQ side a simulated 
high voltage scan of a virtual detector module in- 
side DCS. On the other hand simulated TDAQ 
data is visible in DCS. 

These tools were very helpful during the operation 
and they are now an inherent part of the detector 
control system. 



5. Interface to the Conditions Database 

Conditions data is seen as every data needed for 
reconstruction besides the event data itself, thus 
it has to reflect the conditions the experiment was 
performed and the actual physics data were taken 
[5]. For the pixel DCS this includes basically pa- 
rameters of the detector modules such as voltages, 
currents and temperatures but also parameters 
and status information of further DCS devices. 
As already mentioned, the ATLAS detector con- 
trol system is based on the software PVSS. PVSS 
contains an internal archive to store and to read 
back the history of DCS values, but does not allow 
to access the data from outside the PVSS frame- 
work. 

Therefore a PVSS API^ manager was developed 
by the Lisbon TDAQ group. This custom made 
manager is based on a CH — h interface between 
PVSS and a MySQL database. When running, it 
connects to each of the DCS parameters defined 
by the user and stores the values together with a 
timestamp in the database. When a value change 
occurs, the previous value is updated by replacing 
the timestamp by a time interval and the new 
value is stored in the database in the same way as 
the first value. 

During the combined test beam the system re- 
liability of the pixel DCS set up, the amount of 
data and the handling of the interface have been 
examinated. 

Due to the given storing mechanism described 
above, no data filter or smoothing processes could 
be used. As one result we had about 5 storing pro- 



^ Application Programming Interface 



4 




Fig. 7. Schematic of data flow through the interface 

cesses per second and per value which produced a 
non acceptable amount of data and necessitated a 
limitation of changes. By the integration of a new 
storing mechanism with a storage at the begin of 
a run, every minute ^ and at the end of the run 
we were able to reduce the amount of data signifi- 
cantly. 

Based on about 150 Bytes per storage for a detec- 
tor module, pixel DCS would produce more than 
37 GBytcs of data for physics analysis per year at 
an estimated 5 minutes storage interval. 



6. Summary 

At the combined test beam we have built up a 
pixel detector control system which worked very 
well during the four month beam period. Pixel 
specific software tools were used with good accep- 
tance by shifters. Many functionality issues could 
be studied sufficiently. 

The DAQ-DCS communication software was 
tested intensely and was established very success- 
fully in the pixel configuration. We were able to 
use the full funcitonality of DDC. We provided 
commands for several actions inside DCS. TDAQ 
data were computed by DCS in a well defined way 
while DCS data was used by TDAQ for monitor- 
ing. Messages with severity flags were available. 
From this point all further requirements to pixel 
DCS coming with a system scale up could be 
achieved by this package. 



® If a monitored value run out of its limits this storage 
interval was scaled down to get more information about 
the bahavior 



DDC is the appropriate tool to handle the inter- 
action between on and off-detector parts of our 
optical link. It allows us to develop tuning algo- 
rithms to find the optimal operation point for the 
components of the read out chain. As a first step, 
a graphical user interface which shows inside DCS 
various parameters of the BOC is currently under 
development. 

The used interface to the conditions database did 
not cover all the pixel DCS aims. After the com- 
bined test beam ATLAS intended to use the LHC 
Computing Grid (LCG) framework for develop- 
ing a new interface to the conditions database 
which makes available general database tools and 
interfaces for subsequent analysis. Further better 
cofigurability and more flexibility for filtering data 
as well as the possibility to read data from the 
conditions database in PVSS has to be considered. 
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Fig. 8. Graphical user interface - SIT 




Fig. 9. Graphical user interface - DDC 
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